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INTRODUCTION 


Nationally, more than one million youth drop out of high school each year. One in four young 
people do not graduate with their age mates (Civic Enterprises et al. 2012). Thus, in recent years, 
national leaders have directed sustained attention to what they tenn the “dropout crisis,” 
particularly in high schools that are graduating less than two-thirds of their students. The U.S. 
Department of Education has responded by requiring more transparency about graduation rates 
and providing unprecedented support to address this crisis through programs such as School 
Improvement Grants and the High School Graduation Initiative. 1 

With recognition growing that improving graduation rates is key to our country’s economic and 
civic success, states and local communities have also responded to the dropout crisis. Many 
districts have taken concerted action to support students who are struggling in school and to 
recover young people who have dropped out. Key to the success of these efforts is the creation of 
early warning systems. These systems are based on breakthroughs in understanding the types of 
data analysis that can guide districts and schools to systematically identify youth who are 
struggling or have fallen off track and to put in place support systems and accelerated programs 
that help these young people get back on track and succeed in school. 

The work of building strong early warning systems, which has begun to yield compelling results, 
shows us that dropping out is not an intractable problem. In fact, that work provides strong 
evidence that local and state education agencies can make significant progress when they 
identify students in need of supports early, i.e. in 9 th grade or before, and implement a range of 
whole school improvements, targeted interventions, and specific programs that better address the 
needs of struggling students and put them back on a path to graduation. Systemic dropout 
prevention efforts suggest promising results— as over recent years our nation’s graduation rates 
have increased by three and a half percentage points between 2001 and 2009 (Civic Enterprises 
et al. 2012). But much more work is required to ensure that many more young people complete 
high school with skills to enter postsecondary education and compete in our high-skilled 
economy. 

Along with early warning indicators, another important type of data study is segmentation 
analysis. This study involves looking at data on current high school students who are 
significantly off track for graduation and using that information to plan appropriate recovery 
programming for specific groups of students (e.g., those far off track to graduation but still in 
school). While early warning indicators focus on dropout prevention, segmentation studies help 


1 See: http://www2.ed.gov/programs/dropout/index.html 
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districts assess the numbers of youth who are off track for graduation (and by how much) and 
inform the growth of appropriate recovery programming. 

For many districts (including recipients of High School Graduation Initiative grants and others), 
early warning systems are central to dropout prevention efforts—catching students before they 
drop out. For those frontrunner districts that have begun to conduct and use segmentation studies, 
the two systems operating together can have a significant impact on raising graduation rates. 

With this growing evidence in hand, the Everyone Graduates Center and Jobs for the Future, 
with the U.S. Department of Education, have developed this resource guide for states and school 
districts to help them establish early warning indicators of high school dropout and launch 
segmentation analysis. The goal is to encourage more states and school districts to develop and 
scale up early warning systems, segmentation studies, and related supports, as well as to offer 
direction and advice in developing these data-driven, resource-efficient systems. 

We know from experience with frontrunner districts that this work can result in more responsive 
school environments, more individualized and timely supports, and more effective recovery 
programming. We trust that the information contained in the guide will be useful to the districts 
as they take on and deepen their work, and we welcome feedback on the usefulness of the guide 
in achieving these aims. 

WHAT’S IN THIS GUIDE? 

Early warning systems begin with indicators that predict high school dropout. Establishing a set 
of indicators provides a powerful tool at the K-12 level that can alert educators to students who 
need intervention to stay on track to graduation. Good indicators have a number of key 
characteristics: 

• Based on evidence and research from school districts and states; 

• Include a limited set of data points that are readily available to school districts (and easy 
to collect and report); 

• Effective at predicting which students are likely to dropout; and 

• Efficient at avoiding the identification of students who would graduate without 
supplemental intervention. 

Many school districts and states are now implementing systems that take administrative data 
(e.g., attendance, course grades, and discipline), turn them into early warning indicators, and 
generate reports for school staff, listing those students most at risk of dropping out or failing to 
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graduate. A few leading districts are combining early warning systems with segmentation 
studies. Segmentation studies use data on student age and credits earned to identify students who 
are significantly off track to high school graduation and unlikely to graduate without an 
alternative option. Such data can prove a powerful tool in planning and designing recovery 
programming. 

This technical resource guide leads school districts and their staff (at both administrative and 
school levels) through the technical steps involved in establishing an early warning system and 
conducting a segmentation study. The guide is organized into seven sections. 

• Section 1 provides basic background on early warning systems for staff at all levels, the 
research behind these systems, their uses, and the reasons why a school district would set 
their development as a key goal for its staff and schools. 

• Section 2 is intended primarily for a school district’s department of research, assessment, 
and evaluation. It specifies the type of data needed to establish an early warning system, 
as well as the simple descriptive ways to analyze the data to detennine a set of early 
warning indicators. 

• Section 3 discusses the types of reporting that are essential to an early warning system. It 
provides detail on how district-level data can be captured in user-friendly reports for 
school-level staff who work directly with students. This section is particularly intended 
for use by the district’s technology and information department that generates the reports 
electronically from databases. 

• Section 4 provides practical examples from school districts that developed and 
implemented early warning systems. It also suggests some solutions to the most common 
roadblocks that districts encounter at each stage of developing an early warning system 

• Section 5 walks districts through the rationale for and steps to conduct a segmentation 
study. 

• Appendix A provides some information on additional resources that can help to guide a 
district with the steps needed to develop an early warning system, first ensuring effective 
use of information at the school level and then organizing staff and services to fashion a 
system of effective interventions. Additional resources listed here provide guidance on 
achieving staff buy-in, providing professional development, building an intervention 
team and selecting appropriate interventions for identified students. This infonnation is 
relevant for both district-office and school-based staff. 


7 



Each step involved in developing an early warning system (and doing segmentation study) may 
be directed by different departments or staff within a district (see figure). Yet while each section 
of the guide is directed to staff with key responsibilities for the particular areas of work, it is 
worthwhile for all involved staff to understand what developing an early warning system will 
entail for their district, the different tasks needed to set up the system, and the operational roles 
of the various departments. While a central administration office may initiate the process by 
setting the goal for the district, the active support of all parties is needed to make the 
development and use of the early warning system a high priority within the district. Similarly, it 
is school-level staff who ultimately put the reports created by an early warning system into 
practical use and work with students directly, so they must understand and buy into the 
development of the system and give input throughout the process. It is essential that all involved 
departments and stakeholders are represented in planning meetings from the initial startup. 

Flow Chart for Developing an Early Warning System: Who’s Involved 












1. OVERVIEW OF EARLY WARNING SYSTEMS 


Over the past decade, several developments have merged to give educators and their partners 
both an interest in addressing high dropout rates and the ability to reduce them significantly. 
First, alarmingly low graduation rates in many high schools and districts throughout the nation 
sparked a surge in interest on the part of communities, educational organizations, and 
government organizations at all levels in finding better and more intentional ways to stem the 
numbers of young people leaving high school without a credential. Concurrently, research has 
demonstrated that high school dropouts can be identified early, long before they fall off the path 
to graduation (Allensworth & Easton 2007; Balfanz & Herzog 2007; Neild & Balfanz 2006). 
Using just a few key, readily available data points, school districts and staff can identify those 
students who are most likely to drop out. Combined with a third development—the continued 
improvement of student information systems and databases by school districts and states—we 
can now use such data to create and implement early warning systems that address the dropout 
crisis much more effectively. According to the National Governors Association, at least sixteen 
states and an ever growing number of school districts now use some fonn of early warning tool 
to predict, as early as middle school, the students likely to struggle to graduate on time, so that 
they can target interventions. 

By intervening early with students who are most likely to drop out, school districts can keep 
youth on track to graduating and work toward increasing overall graduation rates. Also, by 
intervening before the actual dropout event, staff can help students get back on track in ways that 
are both easier to do from the perspective of reaching students and cheaper to do when compared 
to the costs of dropout recovery. Early warning systems also help districts allocate their 
resources more efficiently by helping them focus interventions on students who are most likely 
to drop out and spend less on students who would graduate without any additional supports. 

EARLY WARNING INDICATORS AND THEIR USE IN BUILDING EARLY 
WARNING SYSTEMS 

Early warning systems are based on a set of key indicators or flags that identify those students 
most likely to drop out. Good indicators share four characteristics: 

• Empirically created. Indicators are most accurate and powerful when they are based on 
analyses of longitudinal data that track individual student progress over time. In essence, 
indicators use the experience of previous student cohorts to intervene when students in 
current cohorts begin to show behaviors associated with dropping out among their older 
peers. Studies in a number of states and districts and more sophisticated statistical analyses 


9 


conducted by researchers have confirmed the consistency of such indicators in predicting 
who will drop out of high school. 

• Simple and easily collected. Early warning indicators use readily available data that schools 
typically maintain already (e.g., grades, attendance, classroom behavior and disciplinary 
action). 

• Include only a few key variables. A few key indicators are easier for teachers to monitor 
than a large set of predictors. K-12 analyses have demonstrated that although the underlying 
issues that produce a poor grade or weak attendance may be complex and vary from student 
to student, a small number of consistent flags alert educators to a student who is potentially 
falling off track. By extension, a good indicator system also makes clear which variables are 
not strong predictors of dropping out. 

• Efficient and effective. A good set of indicators accurately identifies which students are 
likely to drop out of high school so that additional resources can be focused on preventing 
their dropping out. At the same time, a good set of indicators captures a broad swath of 
students who may eventually become dropouts, avoiding the “1 percent problem”—that is, 
indicators that are highly predictive but only identify a small percentage of potential 
dropouts. 

Dozens of school districts and many states have developed early warning systems, using student 
data to flag those students who are most at risk of dropping out and who can then be identified 
by teams of school staff for intervention. The quick analysis of a few key student data points 
from administrative data bases presented to school staff in clear and concise reports not only 
enables staff to identify the students before they disengage from school but also helps staff 
develop and choose the interventions that are best suited, given the number of students they are 
dealing with and the types of struggles those students are exhibiting. 
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2. DATA EXTRACTION 


WHAT KINDS OF DATA DOES A SCHOOL DISTRICT NEED? 

Typically, determining a set of early warning indicators is a task for a district’s department of 
research, assessment, and evaluation. Its staff have access to and are familiar with stored records 
of student data 2 . Prior research and the experiences of other districts can help guide the selection 
of indicators, but each school district operates in its own context and establishes its own set of 
indicators that best match its unique student population. The selection process begins by 
identifying one group of students and examining their school-related characteristics and high 
school outcomes in close detail. Typically, the district staff selects a group of ninth-grade 
students, although they can choose to analyze a group of middle-grade students if the district 
wishes to intervene with youth before they reach high school. 

The group of ninth-grade students to be studied must be one from several years earlier, making it 
possible to compare data on their school experiences (e.g., attendance, grades) with their high 
school outcomes (e.g., whether they graduated, dropped out, or remain actively enrolled). For 
example, staff might select all ninth-grade students from the 2005-06 school year and a look at 
their high school outcomes by 2008-09 and their expected date of graduation. Studies like this 
that track the same group of students over time are called longitudinal studies. Thus, an early 
warning system will require a longitudinal data system, including a consistent student identifier 
across all years and within the graduation information file. 

An early warning indicator analysis must track students at least to the time of expected 
graduation. If a school district is going to study a group of ninth graders, they will need at least 
four years of data for those students. Wherever possible, though, it is best to follow students up 
to two years past their expected time of graduation. This is because many students take more 
than four years to complete high school; it takes five or six years of data to determine their final 
outcome. However, the student information systems or databases for many school districts do not 
go back far enough to include the extra two years past expected graduation. 

Early warning indicator analyses are also cohort studies: they follow a group of students who all 
started at the same time. However, it is not always possible for districts to identify the full cohort 
of first-time ninth graders and repeaters—data from back years may be missing or may not exist, 

2 It is assumed here any student data used in the development of an early warning system will only be shared 
internally with designated district or school officials having legitimate educational interests. Should any district 
consider sharing their student data with external partners such as local and community organizations that may be 
assisting with development, they must first ensure that the sharing of such student based information is in 
compliance with the Family Educational Rights and Privacy Act (FERPA). More details regarding FERPA regulations 
are available at: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html 
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and some students transfer in from other districts where they may have repeated. Staff should 
exclude repeaters from the cohort when possible, but this is not always possible. As a fallback, 
staff can take the group of students identified as ninth graders in the selected year. 

Once a district decides on the grade level where they want to begin the analysis and identifies the 
years of data needed for tracking the students, it is ready to extract three basic types of data for 
analysis: 

• Demographic: Gender, ethnic background, age, free/reduced-price lunch program 
eligibility, English language learner status, special education status; 

• Academic: Attendance, suspensions and disciplinary actions, course grades, and test 
scores; and 

• Enrollment: Enrollment and withdrawal records. 

School districts already have these types of data in their administrative files and databases. While 
some districts may wish to include additional types of data in their early warning systems out of 
interest, they do not need to collect any new data to complete these analyses. 

The demographic and academic data are only needed from the ninth grade (or other selected 
grade) school year. This information is not needed for every year in which the students will be 
tracked. The demographic and academic indicators from ninth grade are then compared with 
their final high school outcomes to determine which indicators are the strongest predictors of 
whether students graduate or drop out. Only enrollment data are needed from each and every 
year that the students are tracked. Detennining each ninth grader’s final high school outcome 
requires the enrollment/withdrawal data from each year to catch those students who drop out or 
transfer before the expected year of graduation. 

For each school district, the specific data measures may vary depending on what is recorded in 
its student infonnation systems (SIS). For example, if both “days present” and “days absent” are 
recorded, a district can calculate each student’s attendance rate, but some districts only have 
“days absent” available. For suspension data, some districts may have out-of-school suspension 
data and in-school suspensions combined, while others may have them separated, and others still 
may record only the number of days lost due to suspension. 

Course grades are the indicator that varies most across school districts. Ideally, staff can count 
the number of course failures by each student, as well as the number of failures in math or 
English. But some districts only have each student’s total GPA and measure whether he or she 
was failing by looking at an overall average. Similarly, test scores may come from state 
accountability tests, district tests, or end-of-course exams. Whether the goal is to track 
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attendance, behavior, course grades, or test scores, staff select the best measures they can from 
those available in the student records. For student demographic measures, staff include as many 
metrics as possible, though in most cases one or two of those listed above may be unavailable. 

Staff use enrollment and withdrawal records to determine each student’s final high school 
outcome (graduate, dropout, still active). To do this, they determine each student’s final status 
from the most recent drop/withdrawal code. For example, if the sample tracks a group of 
students who were in ninth grade in 2005-06 through to 2010-11 (two years past expected time 
of graduation), then the 2011 drop/withdrawal code determines the final status. For any students 
missing a 2011 withdrawal record, their 2010 record would be used (and so on, going back as far 
as 2006 for students who withdrew during the original ninth-grade year). With the final status 
obtained for each student in the cohort, the local district withdrawal codes are categorized into a 
final outcome: 1 = graduate, 2 = drop out, 3 = still actively enrolled, or 4 = transferred out of the 
district. 

Staff should exclude students categorized as having transferred out of the district from the cohort 
for early warning indicator analyses, as well as students who transfer into the cohort after the 
ninth grade. Unique withdrawal codes that do not easily fit into any of the above four categories 
(e.g., deceased) are assigned in a way that fits most closely with the district/state’s graduation 
accountability policies, in terms of counting a student as a dropout or excluding them as a 
transfer. Similarly, staff may want to categorize students who are still active in their fifth or sixth 
year of high school as dropouts, rather than treat them as still active if relevant accountability 
policies count such students as dropouts. 

Finally, enrollment records can be used to calculate three more indicators of student mobility 
(and calculated only for the student’s ninth-grade year). The number of enrollment records 
during the ninth-grade year can be used to calculate the number of transfers students made 
during that year, while prior enrollment records can be used to determine how many students 
transferred to other schools from the eighth to the ninth grades, as well as how many student 
were new to the district (no prior enrollment records in the school district before the ninth grade). 

DETERMINING THE BEST EARLY WARNING INDICATORS 

With the sample of students defined and the basic data extracted from the administrative student 
records, staff can begin analyzing the data and determining the best set of early warning 
indicators. All of the analyses described here are simple, based only on the calculation of 
percentages. Any school district should be able to replicate them without the aid of outside 
technical support. 
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Before beginning the analyses, it is necessary to prepare the data for analysis. Each of the 
indicators must be turned into a dummy variable (also known as a categorical variable). These 
are variables that have only two values: each student is assigned either a “1” or a “0” depending 
on whether she or he has that specific indicator. For student demographics, this is very 
straightforward: students receive a 1 if they have that characteristic (e.g., special education; 
Hispanic; female) and a 0 if they do not fall into that group. One dummy variable must be made 
for those indicators if they are not already in a 1/0 format. For example, a variable for 
race/ethnicity will require several dummy variables, one for each ethnic category. However, 
variables that only have two categories (e.g., gender; eligible for free/reduced-price lunch) 
require only one dummy variable. 

The academic indicators require several dummy variables for each indicator. For example, to 
prepare student attendance data, staff might create one variable for whether students had 
attendance rates under 90 percent, a second variable for whether their attendance rates were 
under 85 percent, and a third for whether their attendance rates were under 80 percent. The series 
of dummy variables are independent of each other (i.e., a student can be included and coded as 
affirmative in each of the dummy variables if his/her attendance rate is below 80 percent); the 
multiple dummy variables allow school staff to parse the data as needed). Comparisons between 
them enables staff to detennine at which level, or at which cut-off, attendance rates act as the 
best indicator of future high school outcome. For suspensions, they might use one variable for 
one or more suspensions, another for two or more, another for three or more, etc. For course 
grades, a district could create several variables for students failing one or more semester classes, 
two or more, three or more, etc., as well as separate variables for whether a student failed a math 
class and for whether they failed an English class. 

Dummy variables for test scores are often based on performance categories—for example, 
whether students’ scores fell in the basic or below basic categories. Finally, dummy variables for 
high school outcomes need to be made based on students’ final withdrawal status: one dummy 
variable for students who graduated; another for students who dropped out; and one for students 
still actively enrolled in the system. 

As the dummy variables are created and coded, it is important to review the original data and 
check for errors that could lead to assigning an early warning indicator to students incorrectly. 
This could lead to their being incorrectly targeted for an intervention. Most errors will be data- 
entry issues, such as an attendance rate of 90 percent rather 0.90, or out-of-bounds entries such 
as “course grades” of 105 or “days attended” over 180. However, missing data can also be 
problematic if students who are missing attendance data or course grades are automatically and 
mistakenly assigned an indicator due to the way the dummy variables are coded and created. 
Where data are missing, districts should base decisions and selection of the key indicators on 
those students for whom data is available. 
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With the set of dummy variables coded, analysis begins by creating a table based on them (see 
Table 1). For each indicator, the table shows how many students in the cohort fell into that 
category, what percent of all students in the cohort shared that indicator, how many students with 
that indicator dropped out, and for what percent of all dropouts represented in the latter group. 
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Table 1: Dropout Rates by Ninth-grade Indicators 



Characteristic 

Numbers of 
Students with 
Characteristic 

Percent Who 
Dropped Out 
(Efficient) 

Number of 

Actual 

Dropouts 

Percent of 

Total Dropouts 
(Effective) 


ENTIRE COHORT 

6,725 

15% 

979 

100% 


Male 

3,496 

17% 

606 

62% 


Female 

3,229 

12% 

373 

38% 

</) 

o 

Asian 

846 

8% 

7 

1% 

sz 

Q. 

re 

White 

4,623 

13% 

602 

61% 

>_ 

o 

o 

Black 

1,619 

19% 

304 

31% 

E 

Q 

Hispanic 

358 

17% 

60 

6% 

a 

+-» 

Native 

42 

15% 

6 

1% 

<D 

T3 

Over-age 

1,270 

35% 

443 

45% 

D 

<z> 

FRL Eligible 

3,083 

19% 

593 

61% 


Spec. Ed. Status 

886 

21% 

184 

19% 


ELL Status (eligible) 

181 

15% 

34 

3% 

k. 

o 

<90% Attendance 

900 

44% 

393 

40% 

> 

re 

<85% Attendance 

460 

59% 

270 

28% 

.c 

<u 

CQ 

<80%> Attendance 

288 

68% 

194 

20% 

■*-< 

c 

re 

■o 

3 

>1 Suspension 

1,208 

30% 

364 

37% 

>2 Suspensions 

586 

39% 

227 

23% 

W 

>3 Suspensions 

341 

42% 

144 

15% 

Student 

Mobility 

New to School 
District/System 

514 

46% 

238 

24% 

>1 Transfer 

235 

36% 

85 

9% 

>2 Transfers 

23 

42% 

10 

1% 


Failed Math 

167 

41% 

69 

7% 

« 

a> 

1_ 

Failed English 

192 

46% 

88 

9% 

>1 Course Failures 

534 

44% 

235 

24% 

3 

</) 

re 

>2 Course Failures 

303 

55% 

166 

17% 

<u 

>3 Course Failures 

196 

60% 

117 

12% 

o 

£ 

re 

"O 

re 

o 

Below Proficiency in 
Algebra 1 End-of- 
Course Exam 

20 

50% 

10 

1% 

< 

Below Proficiency in 
English 1 End-of- 
Course Exam 

17 

44% 

8 

1% 
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This table enables us to compare the indicators to one another and detennine which best predict 
who will drop out of high school. Look for those indicators that are most efficient and most 
effective. An efficient indicator means that most of the students with that characteristic 
eventually dropped out (“Percent Who Dropped Out”). An effective indicator nets a substantial 
proportion of all dropouts in the cohort (“Percent of Total Dropouts”). To be of practical use in 
an early warning system, an indicator needs to be efficient so that when students are flagged 
resources are not focused on students who would have graduated without additional intervention. 
Indicators need to be effective so that interventions capture a large proportion of the total 
dropouts to stem the problem significantly. 

Finding a balance between efficient and effective indicators is the focus of most decision making 
in selecting the best indicators for an early warning system. The indicators that are most efficient 
are often the least effective. For example, it may be the case that 90 percent of students with 
attendance rates under 70 percent drop out (making it a very efficient indicator), yet few students 
in the cohort likely have attendance rates that low so the indicator might only identify a small 
number of students accounting for only a few percent of all dropouts (meaning it would not be an 
effective indicator). Ideally, we look for indicators where 75 percent of students with that 
characteristic dropped out of high school. While that is the ideal, not all districts will see 
indicators of such strength that are also effective. Thus, it is useful to look for indicators where 
more than 50 percent of the students dropped out (and if possible, closer to 67 percent). 

Similarly, the most effective indicators are often the least efficient, so a balance must be found. 
For example, flagging students who are eligible for the federal free/reduced-price lunch program 
may identify over 50 percent of all dropouts, but if only 20 percent of these students dropped out, 
then the resources spent on intervention would not be used well in four out of five cases. Also, if 
half the students in the district are eligible for free/reduced-price lunch, this would result in 
targeting too many students from a practical perspective of financial and staff resources. 

The total number of students within each indicator is also important to consider. While an 
effective indicator would ideally identify close to 50 percent of all high school dropouts in a 
cohort, those that flag 20 to 33 percent of all dropouts are often the ones selected if they are 
highly efficient and identify a practical number of students for intervention. Working with 20 
percent of all dropouts in a school district would still be addressing a substantial proportion of 
the overall problem. 

Table 1 (above), which synthesizes data from several school districts and states, modified for 
anonymity, reveals some patterns that consistently emerge. One is that demographic indicators 
are far less efficient than academic ones in predicting which students will drop out of high 
school. The exception is students who are over-age for their grade level (defined here as students 
whose age exceeds their grade level by the number 6 at the start of the year—that is, they are 15 
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or older at the start of the ninth grade or 12 or older at the start of the sixth grade). Another 
pattern is that test scores are typically weak indicators for use in an early warning intervention 
system. While they are sometimes an efficient predictor of dropping out of high school, they are 
typically unavailable for many students due to missing test scores and testing variation across 
grade levels. Thus, test scores rarely identify a large proportion of all dropouts, making them an 
ineffective indicator. Test scores are also not available until the end of the school year, but 
attendance rates, course grades, and suspensions can be calculated and used to intervene at any 
point in the school year, and certainly as early as the end of the first marking period. 

COMBINING THE INDICATORS INTO A SET 

Typically, three or four indicators make an ideal set for use in an early warning system. 
Combined, this is enough to identify a group of students that will represent a substantial 
proportion of all dropouts in terms of focusing resources for intervention. The most common 
indicators are attendance, suspensions, and course failures, with the particular thresholds for each 
varying by school district. A few districts also include over-age for grade, which can sometimes 
be a strong indicator that a person will drop out. However, school staff cannot always take action 
on over-age indicators easily, while they can intervene and effect a change in the areas of 
attendance, behavior, and course grades. Similarly, indicators of student mobility are sometimes 
moderate in their strength and may identify students for monitoring, but they typically do not 
bear inclusion in an early warning system. 

Deciding which indicators and cut levels to use is a process; there is no single best level. In 
general, staff select those indicators that stand out as significantly efficient, effective, and 
malleable compared to others. Involving counselors in the selection of these indicators will help 
in determining which are best; it will also assist with the school-level buy-in process. 

Staff should try different combinations, selecting several from and replicating Tables 2 and 3 
below, to gauge their overall power as a set for use in an early warning system. Comparing the 
overall power of different combinations will help determine which set would work best and suit 
the district’s particular goals. 

If resources allow, districts and their staff should revisit these analyses each year to detennine if 
the cut levels require adjustment as the student population shifts. Cut levels will also vary by the 
grade level to which they will be applied: the indicators tend to require looser cut levels (e.g., 
one failure instead of two; 80 percent attendance instead of 90 percent) when applied to middle 
grades, where student disengagement is much less of an issue. 

In general, staff should establish that the selected indicators are common to those students who 
eventually drop out but rare among those who remain on the path to graduation. Table 2 shows 
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the percent of students with each indicator, and with each total number of indicators, comparing 
the rates for the entire cohort to those for dropouts and graduates. In this example, one-third to 
one-half of dropouts had each of the individual indicators, but less than one-fifth of students in 
the entire cohort had them and less than one-tenth of graduates had the indicators. As a combined 
set, more than two-thirds of the entire cohort and more than 80 percent of graduates had none of 
the indicators. In comparison, less than one-third of dropouts had none of the indicators; 70 
percent of dropouts had one or more of the key indicators. 

Table 2: Distribution of Students with Academic Indicators 



Percent of All 
Students 
(N=6,725) with... 

Percent of 
Dropouts 
(N=979) with... 

Percent of 
Graduates 
(N=5,045) with... 

Data on Each Indicator 

<85% Attendance 

16% 

45% 

7% 

>2 Suspensions 

14% 

34% 

8% 

>2 Course Failures 

18% 

46% 

8% 

Number of Indicators 

0 Indicators 

70% 

30% 

83% 

11ndicators 

17% 

30% 

13% 

2 Indicators 

9% 

25% 

4% 

3 Indicators 

5% 

15% 

1% 

1 or more 

30% 

70% 

17% 

2 or more 

13% 

40% 

4% 


Table 3 shows similar rates to those in Table 2, but this type of report helps staff decide the 
appropriate intervention level. Over 80 percent of the students with none of the indicators 
succeeded in graduating, while the graduation rate plummets below 40 percent for students with 
one or more indicators. However, at the 1 indicator level, the ratio of dropouts to graduates is 
still close to even (44 percent vs. 36 percent). For intervention purposes, staff want to ensure that 
an investment of additional resources would make sense for almost all the students identified. 

For students with two or more indicators, the graduation rate falls below 20 percent and the ratio 
of students who dropped out to those who graduated is almost 5 to 1 (64 percent vs. 14 percent). 
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Table 3: Percent Dropped Out or Graduated, by Number of Indicators (total n of students 
= 6,725) 



% Dropped out 

% Graduated 

Number of Students 

0 Indicators 

9% 

82% 

4,708 

1 Indicator 

37% 

44% 

1,143 

2 Indicators 

62% 

15% 

605 

3 Indicators 

73% 

8% 

336 

1 or more 

44% 

36% 

2,084 

2 or more 

64% 

14% 

941 


From a practical perspective, flagging students with two or more indicators in Table 3 would 
result in identifying roughly 14 percent of the total cohort (941 of 6,725 students) for substantial 
and sustained interventions. Given that 64% of those flagged students ended up dropping out 
while 14% graduated, the targeted group would have identified over half of all dropouts in the 
cohort and included only 3 percent of the students who would have graduated anyway. Any 
programs targeting such students would catch the correct students in at least three of five cases 
while addressing a large proportion of the cohort’s dropouts, thus making efficient use of finite 
resources and making a substantial impact on the overall problem. A larger set of students, those 
with a single indicator or students who have other indicators of interest that were not included in 
the set (e.g., over-age, mobility deficits) might be identified for more moderate interventions or 
for monitoring so that intervention can commence if additional signals begin to occur and before 
the students drift too far off track. 

Again, research staff should replicate Tables 2 and 3 with several different sets of indicators and 
different cut levels (e.g., 85 percent attendance vs. 90 percent) to test which combinations to use 
in an early warning system. Districts must also pay attention to the total number of students 
identified for intervention to be sure that intervention is feasible, considering available resources. 

That said, schools may not always be able to work with all students identified by the set of early 
warning indicators but rather may focus on select subgroups of students for staff to work with. 
Schools may choose to work with students who have all three indicators since those students 
have the greatest needs or they may choose to work with those students who have only two 
indicators in order to catch them before they become more fully disengaged from school. Much 
of the decision-making on which of the identified students to target will rest with the school 
leaders and staff/counseling teams who are most knowledgeable about resources and objectives. 
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As a district’s research staff replicates tables such as Table 3 above, they should not expect to 
replicate those exact numbers. Each district will vary in terms of the frequencies of each 
indicator and how strongly they predict future dropout events. Also, districts must be aware that 
the numbers will vary greatly depending on whether the longitudinal data for the cohort runs 
only until the expected time of graduation or if it is extends further out. Because many students 
take longer than four years to graduate and remain actively enrolled past four years, any district 
whose data run only until the expected year of graduation will see numbers that appear much 
weaker than those in Tables 2 and 3, which are based on data two years past expected time of 
graduation. 

Students who remain enrolled past four years of high school are much more similar to dropouts 
than they are to graduates in terms of how frequently they exhibit the key indicators. 
Accordingly, many of those who remain enrolled end up dropping out, especially those who 
exhibit two or more indicators. The effect of having only four years of data is that the group of 
students identified will seem to capture both many fewer dropouts and many more students who 
remained actively enrolled. Where staff are selecting indicators based on only four years of data, 
they should keep this effect in mind and select indicators based on their relative strengths in 
tenns of the results they see, knowing that the same set of indicators would appear more efficient 
and effective given five or six years of data. 

While these methods of analysis rely on simple averages, the results from districts and states that 
have completed the process have been tested using more sophisticated methods of statistical 
analyses (EGC, 2010a; 2010b; 2010c; 2011). Districts can use results based on these 
comparisons of averages that they can conduct themselves, knowing that more complex 
methodologies would yield similar results. States and districts have also found that the key 
indicators (typically attendance, course failures, and suspensions) operate equally as well for 
urban and rural districts or large and small ones (EGC, 2010b). Despite minor variations, the 
indicators operate with similar strengths and patterns across all types of districts (EGC 2010a; 
2011). Finally, while early warning indicators have worked when applied to earlier grades, their 
predictive power may prove somewhat weaker than in ninth grade (EGC, 2010c). 
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3. REPORTING 


CREATING EARLY WARNING SYSTEM REPORTS 

The next challenge is to generate reports for schools that identify students flagged for 
intervention by school staff. This is usually a matter of working with the electronic databases or 
student information systems to produce the desired reports, so this step often involves a district’s 
information technology (IT) department. While these lists are generally quite simple, producing 
them can often be one of the most difficult roadblocks in implementing an early warning system. 
For some districts, their data are stored in an SIS that can easily generate customized reports for 
teachers and school staff. However, the SIS databases in some districts cannot be accessed or 
managed easily, and producing the lists can be quite problematic. 

Typically, the IT department and the district office of research and assessment collaborate 
closely to obtain access to all required data and develop the reports. Coordination with school- 
level staff is also important: they are the end users of the reports, and they are often the original 
source for the relevant student data. Districts also can contact the vendors and developers of their 
SIS platforms, who are often eager to improve their products to make them more attractive to 
school districts and might be willing to make customized modifications. 

Table 4 is a sample spreadsheet report. Color-coding students based on the degree to which they 
are off track is a common tool to help school-level staff easily distinguish among students. These 
lists might also include additional information on each student, such as personal or demographic 
information, notes on his or her history, or other information that helps develop a student profile. 
However, lists that contain too much information are often overwhelming and counterproductive. 
A key goal is to keep reports as simple, clear, and concise as possible. Once school-level staff 
decide which students to work with, they may decide to look for more detailed data on those 
specific students 3 . 

Reports should be generated as early and as often as possible, and students at risk for dropping 
out should be identified early. Where possible, school staff should receive reports at the 
beginning of the school year that are based on the prior year’s final data (e.g., last year’s eighth- 
grade data for students at the start of ninth grade). This enables them to identify those students 
most likely to need extra support before they have started to fall behind in school. Throughout 

3 Again, it's important to note that early warning indicator reports based upon student data should only be shared 
internally with designated district or school officials having legitimate educational interests. Should any district 
consider sharing their reports with external partners such as local and community organizations that assist with 
intervention efforts, they must first ensure that the sharing of such student based information is in compliance 
with the Family Educational Rights and Privacy Act (FERPA). More details regarding FERPA regulations are available 
at: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html 
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the year, reports could be made available biweekly, but they should be generated at the least at 
the end of each marking period as data become available. For example, students can be flagged 
for low attendance rates as early as the first two weeks of school if the number of absences puts 
them on pace to hit the district’s selected indicator level for the full year. Students can be flagged 
for their first quarter course grades if they are failing even if credit/failure is not assigned until 
the end of the semester. And students who receive a suspension early in the year might be 
flagged so they can receive early academic or behavior support. 

As noted, not all students identified by the lists are likely to be selected for intervention, and 
decisions on which students to target are likely to be made by school-level staff based on 
available time and resources, as well as on each individual student’s background and case. 


Table 4. Sample EWI Data Sheet Provided to Classroom Teachers 


Student 

Name 

Attendance 

Indicator 

Attendance 

Rate 

Course 

Indicator 

Course 

Failures 

Math 

Mark 

Reading 

Mark 

Suspension 

Indicator 

Suspensions 

TOTAL 

Indicators 


Student Z 

1 

83 

0 

0 

D 

C 

1 

2 

2 

Student X 

1 

81 

1 

2 

F 

F 

0 

0 

2 

Student J 

0 

87 

1 

2 

F 

F 

1 

2 

2 

Student Y 

1 

82 

0 

0 

D 

D 

0 

1 

1 

Student W 

1 

80 

0 

0 

C 

B 

0 

0 

1 

Student K 

1 

85 

0 

0 

B 

D 

0 

0 

1 

Student T 

0 

98 

0 

0 

B 

A 

1 

3 

1 

Student E 

0 

92 

1 

2 

F 

F 

0 

1 

1 

Student O 

0 

93 

0 

1 

C 

F 

0 

1 

0 

Student H 

0 

90 

0 

1 

F 

B 

0 

0 

0 

Student N 

0 

92 

0 

1 

D 

F 

0 

0 

0 

Student D 

0 

93 

0 

1 

F 

D 

0 

0 

0 

Student S 

0 

97 

0 

1 

F 

C 

0 

0 

0 

Student C 

0 

89 

0 

0 

A 

C 

0 

0 

0 

Student M 

0 

89 

0 

0 

C 

A 

0 

0 

0 

Student B 

0 

90 

0 

0 

A 

A 

0 

0 

0 

Student G 

0 

90 

0 

0 

C 

D 

0 

0 

0 

Student F 

0 

91 

0 

0 

D 

B 

0 

0 

0 

Student L 

0 

91 

0 

0 

A 

C 

0 

0 

0 

Student P 

0 

94 

0 

0 

C 

A 

0 

0 

0 

Student A 

0 

95 

0 

0 

C 

B 

0 

0 

0 

Student Q 

0 

95 

0 

0 

B 

C 

0 

0 

0 

Student R 

0 

96 

0 

0 

C 

C 

0 

0 

0 

Student U 

0 

99 

0 

0 

A 

B 

0 

0 

0 

Student V 

0 

100 

0 

0 

B 

B 

0 

0 

0 
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4. ILLUSTRATIVE EXAMPLES 


A number of school districts have completed the process outlined in this guide to develop early 
warning systems. Each of the four districts described in this section has worked within its distinct 
context, in terms of the student population it serves, its region of the country, and its resultant 
policy differences. As such, the districts’ processes and end products are unique, first by the 
specific indicators and cut-off levels each district selected and also by the types of roadblocks 
staff encountered. 

Nashville, Tennessee, was one of the first districts to develop an early warning system. It made 
an initial proposal to create early warning indicators in fall 2009, and it began analyzing data in 
spring 2010 to determine the best set of indicators. Planning meetings took place during the 
summer and fall of 2010, and the district rolled out the system that winter. 

Implementation went smoothly in this district, in part because every monthly meeting from the 
beginning included representatives of district IT staff, research staff, and the district’s head of 
counseling. Establishing an early warning system was a long-term project, and its successful 
implementation was achieved largely by ensuring that the process was a collaborative and 
communicative effort among all departments, with each contributing expertise toward the end 
goal. 

If they had to do the project over again, district staff report that they might engage more fully the 
department that oversees schools and principals to make sure school leaders fully understood the 
tool and had common goals and expectations for its use at the school-level. This would ensure 
that more schools used the tool effectively to target interventions. 

St. Clair County Public Schools, Illinois, a regional office, developed an early warning system 
for the three school districts it oversees. All three had graduation rates lower than the state 
average. Regional staff followed the simple, straightforward steps outlined in this guide. First, 
during the 2011-12 school year, they identified a previous cohort of first-time ninth-grade 
students using a longitudinal database of student records. They compared the graduation 
outcomes of those students to their ninth-grade attendance levels, numbers of suspensions, and 
numbers of course failures. Once the ideal cut-off levels had been identified for each indicator, 
they combined them into a set. 

During the following 2012-13 school year, the principal and staff at one high school piloted the 
early warning system. The plan is to expand it to all three school districts next year, with 
modifications based on experiences from the pilot year. 
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In starting its early warning system, Clark County, Nevada, found that it had a large amount of 
data available from their student infonnation system, but the early warning system reports often 
overwhelmed school staff. Too much information negatively affected staff usage. While the 
district had enough data to include five to seven indicators on each student (depending on 
specific grade level), many of these indicators were highly related to one another, and the reports 
generated for school staff were confusing and required a lot of interpretation. 

By eliminating some of these overlapping indicators, the district streamlined the reports and 
made them more concise and easier for staff to work with and use. For example, the reports 
dropped an indicator for students repeating a grade; “over-age for grade” was already included. 
Similarly, course behavior marks were dropped in favor of suspensions as the single indicator of 
discipline and behavior. GPA was cut, while course failures were kept. In each case, the two 
indicators were largely the same, but those with the stronger relationship to student outcomes 
were selected for use. 

Selecting fewer variables created the added improvement of reducing the number of students 
identified to a size that could be realistically managed by the district’s school-level social 
workers. To make the reports even more useable, the district reformatted them to list the 
indicators in order of priority. Listed first were those with the strongest relationships to student 
outcomes (e.g., attendance, course failures, and suspensions). 

By contrast, some districts have developed more intricate early warning systems that include 
more than 304 indicators. The Adams 12 Five Star School District, Colorado, has an early 
warning system that is far more intricate than most. While it includes the categories of 
academics, attendance, and behavior, it also includes a social “risk factors” category based on 
information from a survey administered by the district to incoming freshmen. Further, each of 
the four categories is made up of several individual indicators. Academics include course failures 
and GPA, credits accumulated, and test scores. Attendance includes both unexcused and excused 
absences and truancy. Behavior includes out-of-school and in-school suspensions and 
expulsions. The social risk factor category includes measures for English language proficiency, 
homelessness, and student mobility between schools. 

Each category counts for 100 points, with a maximum total score of 400 for a student. At the 
same time, each category is weighted primarily on those core indicators that analyses have 
detennined to be the strongest: academics is heavily weighted toward course failures; attendance, 
to unexcused absences; and behavior, to out-of-school suspensions and expulsion. This makes 
Adams 12 indicators both unique and based upon solid empirical evidence. 
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5. SEGEMENTATION ANALYSIS 


WHAT IS SEGMENTATION ANALYSIS? 

Segmentation studies identify students who are considerably off track for graduation, typically 
defined as two or more years off track to graduate based on age and credits. These youth are 
highly unlikely to graduate without an alternative educational option. By carefully analyzing this 
population of high school students, districts can use the lens of how far students are from 
graduation, combined with their ages, to determine the best programming options to recover and 
graduate these young people. 

Also, segmentation analyses can be extended beyond a district’s in-school high school 
population to include recent dropouts. This approach enables a district to identify young people 
who dropped out but were close to graduation—for example, those who were short by only a few 
credits or had not fulfilled a state exit exam requirement. By offering programming designed to 
move these students quickly to the finish line, districts can often get a bump up in their 
graduation rates. 

Underlying segmentation study is the assumption that age and distance to graduation matter in 
determining what types of programming are both cost efficient and effective in recapturing off 
track and out-of-school youth. A young person’s age and how close or far she or he is from a 
high school diploma has implications for school schedules, curricula, and staffing. For example, 
a student who is older and close to graduation will benefit from flexible scheduling and 
opportunities to earn credits quickly. A student who is younger and much further from 
graduation can have up to three years before aging out of the K-12 system and will benefit from 
full-time, extended day and summer programming, with a curriculum organized to facilitate 
credit accumulation and the acquisition of college-ready skills. 

Unlike studies of early warning indicators, segmentation analyses are not longitudinal and do not 
track the same group of students over time. Instead, segmentation studies take a snapshot of a 
defined population of students at a certain point in time. These types of analyses are called cross- 
sectional studies. They use descriptive statistics and do not require complex statistical analyses. 

DEFINING THE POPULATION 

The first step is deciding the population of students the district will include in its segmentation 
analysis. A district has to decide on the scope of the population to include. For example, is the 
district interested in segmenting all current high school students, the current high school 
population plus recent dropouts, only recent dropouts, or the student populations of selected high 
schools? 
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Usually, a district’s segmentation analysis looks at both the current high school population and 
recent dropouts. Such an approach enables a district to intervene with students who are in school 
but so off track they are unlikely to graduate without a different option and to recover recent 
dropouts who have similar programming needs. However, fiscal constraints often lead districts to 
limit their analysis to a more defined group of students (e.g., only recent dropouts, only the 
student population of specific high schools). In these cases, districts can define the population for 
analysis based on their priorities. For example, is the district’s primary goal to recover recent 
dropouts and place them in programs matched to their needs? Is it to identify off-track students 
are who are still in school but unlikely to graduate without a different option? 

A district may also want to focus its segmentation analysis on low-performing high schools, 
which often have high concentrations of off track students. Such an analysis can help district and 
school leaders get a better handle on the types of programming needed to turn the school around 
and improve outcomes for students. 

DEFINING THE SEGMENTS 

The next step is to define the segments for analysis. In New York City’s groundbreaking 
segmentation analysis, the district and The Parthenon Group (a national advisory group) defined 
three broad categories: young and far from graduation; old and close to graduation; and old and 
far from graduation (see Table 5). 
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Table 5: Defining the Segments 



Age 

Distance to graduation 1 

Young & 

17 or younger at program 

Enough credits/skills to graduate in two to three 

Far 

entry 

years 

Old & 

Close 

Age 18 or older 

Enough skills/credits to graduate in one year 

Old & Far 

Age 18 or older 

Few credits, two or more years from graduation 


As other districts have taken up segmentation analyses, they have adapted these categories to 
better align with their conditions and the goals of their segmentation studies. Table 6 shows how 
one district refined the categories based on its credit system and added additional ones to 
represent its population of high school students and recent dropouts. 4 

Table 6: One District’s Adaptation of Defining the Segments 



Age 

Distance to graduation j 

Young and Far 

17 or younger at program 
entry 

Enough credits/skills to graduate in two to 
three years 

11 credits or fewer 

Old and Close 

Age 18 or older 

Enough skills/credits to graduate in one year 

12 credits or more 

Old and Far 

Age 18 or older 

Few credits, two or more years from 
graduation, at least an 8 th grade reading level 

11 credits or fewer 

Old and Very Far 

Age 18 or older 

Few credits, two or more years from 
graduation and below an eighth-grade reading 
level 

Over-age Late 
Entrant English 
Language 

Learner 

Age 18 or older 

English language learners who entered the 
school system during high school with limited 
schooling and low literacy in any language 


Districts may decide to expand a segmentation analysis to include students who are less than two 
years off track. These students may not need an alternative option but are at risk of falling further 
behind. At minimum, such students would benefit from close monitoring and may require a 
school-based intervention. Table 7 offers the segments used by a district that conducted such an 
analysis. 


4 Tables 6 and 7 are based on work done by the Denver Public Schools in collaboration with Jobs for the Future. 
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Old Young 


Table 7: Categories of At-risk and Off track Students, by Credits and Age at the End of the 
School Year 


Far 


14/15 


9th Grade 

At-risk 

(<1 year behind) 


10th Grade 


On track 


Near 


11th Grade 


15/16 


16/17 


17/18 


18+ 


Seriously off track At risk 
(1 -2 years behind) (< 1 year behind) 



On track 
At risk 

(<1 year behind) 
Seriously off track 
(1-2 years behind) 

Seriously off track 
(1-2 years behind) 


12th Grade 


On track 
At risk 

<1 year behind 

At risk and old 
(<1 year behind) 


Finally, the district has to determine the extent to which it wants to parse the data. For example, 
does it want to look at the data for each high school? By language background? By 
region/neighborhood? Again, the district can use its priorities and goals for the analysis to 
establish the level of analysis it wishes to conduct. 


EXTRACTING THE DATA 


As with early warning indicator studies, the types of data required for a segmentation analysis 
are typically available to the school district in their administrative files and databases. To 
conduct the segmentation analysis, districts need access to only two types of individual student 
data: age and credits accumulated. Because grade designation in high school varies widely 
across districts depending on promotion policies, we recommend that districts assess how far 
students are off track to an on-time graduation based on age and credits only. 

Data on the age of students are generally straightforward and accessible. However, missing data 
on credit accumulation can present challenges. Students who transfer in and out of the district 
(and/or within high schools in the districts), are late entrants into the district high school, and/or 
have dropped out and then reenrolled can all have considerable gaps in their credit history and 
records may not be easily accessible for these youth. 
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Course grades can sometimes serve as a proxy for credits, but they also vary considerably by 
school districts. One approach to addressing missing credit data is to track individual student 
transcripts at the school level (which are sometimes still found in paper files). This may require 
more resources and staff but can be a useful strategy if the segmentation analysis targets only a 
few high schools. For example, analyzing which segments of students are enrolled in a district’s 
alternative schools and which groups of students are succeeding can infonn efforts to improve 
student outcomes at alternative schools. 

Basically, a district uses the best data available from its student records, with the understanding 
that missing data can result in an underrepresentation in the number and percentage of students, 
whether within the district or in certain schools, who are off track to a high school graduation. 
Missing data can also curtail the level of analysis that the district can conduct. 

Table 8 shows the results of a completed segmentation analysis for a mid-size urban district. 

Table 8: Results from a Segmentation Analysis 


Segment 

Number of 
Students 

Annual 
Dropout Rate 

1: 7 th and 8 th graders demonstrating key risk factors (from 
district’s own early warning indicators study or established 
research) 

NA 


2a. 9 th graders who are slipping: Demonstrating key risk 
factors but are still less than three credits offtrack 

793 

2% 

2b. 9 th graders who are falling: Have risk factors and are 
more than three credits behind 

1309 

13% 

3. Young and far: Age 16-17, and two or more years offtrack 

886 

37% 

4a. Old and very close: Age 17 and less than three credits 
off track 

559 

12% 

4b. Old and close enough to graduate, but maybe not 
with their class: Age 17+ and three to twelve credits off track 

1,529 

25% 

5. Old and far: Age 18+ and two or more years offtrack 

620 

41% 

On track 

11,263 

0.4% 

Percent of all high schools students who are off track 

33% 


Percent of off track students approximately two or more 
years off track 

18% 
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HOW CAN DISTRICTS USE THE RESULTS OF SEGMENTATION ANALYSIS 
TO GUIDE PROGRAMMING? 


A completed segmentation analysis enables district and school leaders to identify the student 
subgroups that are unlikely to graduate without an intervention. The results of a segmentation 
analysis can provide guidance to district and school leaders concerning what types of 
interventions and programming different segments need. For example, districts can use these 
data to map the current demand for options against the district’s supply of alternative 
programming. The analyses of the different segments enable districts to use resources 
strategically to redesign existing options or create new ones tailored to the specific needs of each 
group, based on age and distance from high school graduation. 

Table 9 demonstrates how one district has used segmentation analysis to identify the types of 
interventions needed for students based on age and distance from graduation. 

Table 9: Proposed Interventions for Each Segment 


Segment 

Suggested Intervention needed 

Annual 
Dropout Rate 

1.7 th and 8 th graders demonstrating 
key risk factors (from district’s own early 
warning indicators study or established 
research) 

Very clear plan for transition to 
proper school, likely traditional 
school 

N/A 

2a. 9 th graders who are slipping: 

Demonstrating key risk factors but are still 
less than three credits offtrack 

Current school: Assessment and 
intervention needed midway 
through semester 1 

2% 

2b. 9 th graders who are failing: Have 
risk factors and are more than three 
credits behind 

Current school: Assessment 
needed at semester 

Alternative settings: A portion will 
need alternate setting 

13% 

3. Young and far: Age 16-17 and two or 
more years off track 

Alternative settings 

37% 

4a. Old and very close: Age 17 and less 
than three credits off track 

Finish at current school: Clear plan 
for each student 

Some may need alternative 
setting 

12% 

4b. Old and close enough to graduate, 
but maybe not with their class: Age 17+ 

and three to twelve credits off track) 

Finish at current school, though 
likely with later class 

Alternative setting 

25% 

5. Old and far: 18+ and two or more 

Alternative setting 

41% 


years off track 
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DESIGNING PROGRAMMING BASED ON AGE AND DISTANCE TO 
GRADUATION 


A segmentation analysis enables a district to use resources more strategically in designing 
programming based on how close or far young people are from graduation. Table 10 shows the 
key design components for the three major groups of off track students. 5 Underpinning all of the 
designs is a culture in which advancing to postsecondary education is the norm, along with 
curriculum and an instructional approach focused on postsecondary and career readiness. 

Table 10: Key Design Elements 



School/Program Model 

Curriculum/Programming 

Young and 

• Diploma-granting high school 

• 

Organized to facilitate credit 

Far 

• Full-time, extended-day, and 
summer program to accelerate 

• 

accumulation (e.g., modules, 
competency based) 

Multiyear sequence of career 
and postsecondary exploration 

Old and Close 

• Diploma-granting schools or 
GED programs with clear 
pathways and supports to 

• 

Interdisciplinary curricula that 
meets multiple credit 
requirements and/or 


postsecondary success 
• Flexible programming that 

• 

Self-paced academics with 
strategic use of online courses 


accommodates adult 
responsibilities with enough 
time to get college and career 
ready 

• 

Rapid connection to 
postsecondary 

Old and Far 

• GED programs with clear 
pathways and supports to 

• 

Intensive literacy and math 
catch-up 


postsecondary success 
• Flexible programming that 
accommodates adult 
responsibilities with enough 
time to get college and career 
ready 

• 

Transparent sequence of pre- 
GED academic skill 
development followed by GED 
prep coursework accompanied 
by preparation for credit¬ 
bearing postsecondary 
courses 


Districts have used the results of their segmentation studies to guide decisions on investments in 
recovery programming. For example, in Mobile, Alabama, a local community foundation 
collaborated with the public schools to launch and conduct the segmentation study. On 
completion of the analysis, the district and its community partners launched a school for “old and 
close” students, modeling it on New York City’s Young Adult Borough Centers. They later 


5 Jobs for the Future articulated the design elements in partnership with a number of districts and schools from 
around the country. 
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expanded this model to include a pathway for old and close youth who had been out of school for 
a number of years. Youth in this school complete the coursework required for graduation and 
transitioning into postsecondary education. Recently, the partners opened a Diploma Plus school 
for its young and far population. 

After completing its segmentation analysis, the Denver Public Schools developed a strategic plan 
not only to align its existing Intensive Pathways to better serve the students identified in the 
analysis but also to develop new schools for those students with the fewest options. The resulting 
plan refocuses the district’s Multiple Pathways Centers to serve the young and far population. 
Engagement Centers serve the old and close population. GED-to-postsecondary programming 
serves youth who are older and farther from graduation. The district’s Call for Quality Schools 
(an RFP for new school development) specifically asked for new school proposals that would 
serve these specific populations. At the same time, the district adopted a weighted student 
funding model for all Intensive Pathway programs and revised its School Performance 
Framework to provide stronger weights for growth measures, such as academic skill gains. 
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CONCLUSION 


This guide details the work required to perform data studies, launch early warning systems, and 
use data results to fashion highly responsive dropout prevention and recovery strategies and 
program options. The guide is both a road map and a toolkit. The work is compelling, informed 
by the work of many districts that moved boldly to lower dropout rates and raise both academic 
achievement and high school completion rates. 

Of course, only a few frontrunner districts have a robust complement of dropout prevention, 
intervention, and recovery services in place or at scale to make maximum impact on the drop out 
issue. Nevertheless, the efforts of many districts represent a solid, impressive start in better 
meeting the academic, developmental, social, and emotional needs of diverse student bodies. In 
using data and fonnulating responsive systems, schools become more responsive learning 
environments, and step-by-step address our national dropout crisis by increasing graduation rates 
that help ensure the continuing economic and civic health of our country. 

Much more needs to be done. Additionally, with states and districts under continued pressure to 
do more with less, it has become increasingly critical that staff use limited resources strategically 
to attain the best possible outcomes for their students—and this includes improving high school 
graduation rates. By identifying early warning signs and addressing them quickly, districts can 
keep more students on track to an on-time graduation. By carefully analyzing the population of 
students who are significantly off track to graduation and designing programming based on these 
results, districts can recover and graduate many more young people. Taken together, early 
warning and segmentation analyses enable the leaders of a district and its schools to develop a 
comprehensive strategy. These data studies help districts focus their resources to achieve the 
greatest impact in helping youth stay on track or get back on track to a high school diploma and 
ready themselves to take the next steps beyond high school. 
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APPENDIX A 


This guide focuses on the technical steps that districts go through to develop a set of early 
warning indicators. Several other resources available from the Everyone Graduates Center at 
Johns Hopkins University and the U.S. Department of Education address early warning systems 
from the implementation side: 


• Learning What it Takes: An Initial Look at How Schools Are Using Early Warning 
Indicator Data and Collaborative Response Teams to Keep All Students on Track to 
Success 

• Team Playbook: Using Data to Keep All Students on Track to Graduation 

• On Track for Success: The Use of Early Warning Indicator and Intervention Systems to 
Build a Grad Nation 

• Mid-Atlantic Equity Center’s Beyond the Indicators: An Integrated Approach to 
School-Level Dropout Prevention 

(http://www.maine.gov/education/speced/tools/b2dropout/reports/indicators.pdf) 

• National High School Center’s Approaches to Dropout Prevention: Heading Early 
Warning Signs with Appropriate Interventions 

(http://www.betterhighschools.org/docs/NHSC_ApproachestoDropoutPrevention.pdf 

• Using Data to Build Early Warning Systems - Department of Education School 
Turnaround Learning Community (STLC) Webinar slides 

(http://www.schoolturnaroundsupport.org/document/using-data-build-early-waming- 

systems) 

(Recording of event at http://vimeo.com/37739265) 

• Utilizing Early Warning System Data to Support Strategic Student Interventions - 
Department of Education 

(http://www.schoolturnaroundsupport.org/document/utilizing-early-waming-system-data- 

design-strategic-student-interventions) 

(Recording of event at http://vimeo.com/37739265) 
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These companion publications provide detail on professional development and the nature of the 
teacher teams that support early warning systems at the school level, as well as some 
implementation examples and experiences from districts that have established these systems. 

The guides also provide ideas on the range of possible interventions. While identifying those 
students most likely to drop out is a necessary first step, the larger question for school staff is 
what to do with this infonnation and how to design interventions that are effective in addressing 
the various student needs. If most of a district’s struggling students are concentrated within only 
a few schools, then those schools may best address the problem with whole-school reform 
models or classroom-level interventions. By contrast, schools in which only a few students are 
flagged may require the dedicated attention of one or two staff members and better connection 
with available services and resources. 

These additional resources provide examples of interventions at the student, classroom, and 
school levels, and also by the specific type of indicator problem being faced (e.g., attendance, 
behavior, course failures). 
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